Tracing Text Provenance via Context-Aware Lexical Substitution

Text content created by humans or language models is often stolen misused adversaries. Tracing text provenance can help claim the ownership of identify malicious users who distribute misleading like machine-generated fake news. There have been some attempts to achieve this, mainly based on watermarking techniques. Specifically, traditional methods embed watermarks slightly altering format line spacing and font, which, however, are fragile cross-media transmissions OCR. Considering natural represent replacing words in original sentences with synonyms from handcrafted lexical resources (e.g., WordNet), but they do not consider substitution’s impact overall sentence's meaning. Recently, a transformer-based network was proposed modifying unobtrusive function words), which also impair logical semantic coherence. Besides, one well-trained fails other different types content. To address limitations mentioned above, we propose scheme context-aware substitution (LS). employ BERT suggest LS candidates inferring relatedness between sentence. Based selection strategy terms synchronicity substitutability further designed test whether word exactly suitable for carrying watermark signal. Extensive experiments demonstrate that, under both objective subjective metrics, our well preserve integrity has better transferability than existing methods. approach outperforms state-of-the-art Stanford Word Substitution Benchmark.